Imagine you’re a safety engineer at one of the world’s most powerful artificial intelligence labs, and you’ve just realized that the next generation of AI could be too smart, too fast, and too unpredictable to be released without some serious guardrails. You want to call colleagues at rival companies and agree: let’s all take a breath, slow down, and make sure this is safe before we charge ahead. But then a lawyer taps you on the shoulder and whispers: don’t do that. That phone call could be illegal. That’s the strange, uncomfortable place OpenAI now finds itself in. According to people close to the company, OpenAI has spent recent weeks asking members of Congress for clear guidance about whether orchestrating an industry-wide slowdown on frontier AI development would actually be legal under American antitrust law. The concern is not hypothetical: if multiple AI labs coordinate to delay releases or limit capabilities, that could look like an illegal cartel—companies colluding to restrict output, fix prices, or carve up a market. OpenAI and other major labs may genuinely want to cooperate on safety, but they worry that doing so through formal agreements could expose them to lawsuits, government investigations, and billion-dollar penalties. The result is a bizarre paradox: the very laws designed to protect competition may be preventing companies from cooperating to protect humanity from the risks of their own technology.
At the heart of this mess is a fundamental question about what “safety coordination” really means. Last weekend, OpenAI’s chief scientist, Jakub Pachocki, published a blog post arguing that the future of AI research depends on “coordinating to slow down future development,” especially when it comes to self-improving systems that could learn to rewrite their own code. In the short term, he wrote, he expects “voluntary slowdowns to become commonplace until shared safety bars are established.” In other words, the people building these models increasingly believe that the industry should hit the brakes—at least for a little while—until everyone agrees on minimum thresholds for safety testing, interpretability, and emergency response. But legal scholars are quick to point out that a coordinated slowdown is, from a certain legal perspective, a coordinated restriction on output. That’s a red flag under the Sherman Antitrust Act, the century-old law that was originally designed to break up monopolies like Standard Oil. Nicholas Felstead, assistant director of the Australian Competition and Consumer Commission and a former AI policy fellow at the Center for Law & AI Risk, made this exact argument in a March article. A coordinated pause in AI development, he wrote, could amount to companies jointly deciding to limit the supply of a product—which is one of the classic behaviors antitrust law was designed to prevent. He added, however, that whether any particular agreement would violate the law depends “entirely on the precise details of any agreement.” But even if most safety collaborations would ultimately survive legal scrutiny, Felstead noted, “legal uncertainty can act as a powerful deterrent.” That’s a nice way of saying: no one wants to be the test case, so no one acts.
If you’re waiting for Congress to ride in and save the day, there are early signs that lawmakers are at least starting to pay attention. In July, a bipartisan, bicameral group of lawmakers introduced a bill with a mouthful of a name: the Collaboration on Adversarial Threats and Security Risks Act, or CATSR Act. The bill would explicitly allow AI labs to coordinate on security and safety work without being slapped with antitrust violations. Think of it as a narrow legal safe harbor: you can still compete like crazy on features, marketing, and commercial deals, but if you’re sharing information about a dangerous security incident or agreeing on a basic safety protocol, you shouldn’t need to worry about federal prosecutors treating you like a cocaine cartel. The House version of the bill has been referred to the Judiciary Committee, but it hasn’t been taken up yet. Caleb Knapp, director of government affairs at the nonprofit AI Policy Network, which endorsed the bill, says the legislation would create the legal channels needed for AI labs to work together on safety and security incidents. He also says Congress has a “growing appetite to get something done” on AI safety. But he cautions that actually enacting anything into law may have to wait until after the upcoming midterm elections. That’s Washington-speak for: don’t hold your breath. The bill might be a good first step, but it is still just a piece of paper in a very long, slow process. In an ideal world, you’d want clear rules before the technology gets dangerous, not after. But the world of AI policy rarely moves at the speed of the technology it’s trying to regulate.
Not everyone, though, is convinced that antitrust is the real villain in this story. There is a growing camp of AI leaders who argue that all this hand-wringing about legal liability is simply convenient cover—a way to avoid talking about the uncomfortable reasons why AI developers are wary of collaborating with each other. For one thing, AI is not some academic backwater anymore; it is a massive business, and the companies building frontier models are fiercely competing to capture a new, trillion-dollar market. Sharing safety research is easy in theory, but it becomes a lot harder when you’re trying to sell the same customers the same product at the same time. Some executives also share the Trump administration’s view that staying ahead of China in AI is a matter of national security. In that worldview, slowing down for safety might feel like unilateral disarmament. And perhaps most importantly, different developers have vastly different opinions about what safe AI actually looks like. Some favor heavy oversight, interpretability tools, and conservative release schedules. Others believe the best way to build safe AI is to make it smarter, faster, and more capable, because they think the dangers come from limitations, not intelligence. When your competitors genuinely believe their approach is the safe one, it’s hard to agree on a shared “safety bar” about anything—not because of antitrust law, but because you don’t trust their judgment. John Schulman, an OpenAI cofounder who now serves as chief scientist at rival AI lab Thinking Machines, made this point bluntly on X this week. “First step is for industry leaders OpenAI and Anthropic to stop feuding and work on a pacing proposal together,” he wrote. “They’ll cite antitrust, but that’s fake—antitrust prohibits certain agreements, but not from jointly developing a proposal.” In other words, you don’t need a law degree to write a white paper or host a joint workshop. If you want to collaborate, you can collaborate. But if you want to avoid collaboration, there’s no shortage of excuses.
Meanwhile, the pressure to act has never felt more urgent. Long-simmering fears about the race among AI companies to build and release ever more powerful models exploded into the national spotlight this summer. This week, Jacob Coxon, a former researcher at Anthropic and OpenAI, issued a stark public warning that AI developers were putting humanity at risk—the kind of warning that used to sound like science fiction but now sounds like an all-too-plausible headline. And it’s not just individuals raising alarms. In recent months, a string of security incidents has made the gap between AI’s capabilities and the industry’s safeguards impossible to ignore. At one point, OpenAI’s own AI agents hacked into Hugging Face, another major AI platform, in what was supposed to be a test of the system’s abilities. It worked exactly as designed, which was exactly the problem: the safeguards didn’t keep up with what the model could do. These incidents have led many lawmakers to make urgent calls for AI regulation, but they’ve also deepened the industry’s dilemma. If you try to coordinate with your competitors to prevent the next breach, you risk being accused of collusion. If you don’t coordinate, you risk the next breach being much, much worse. The public is caught in the middle, watching as a handful of companies decide the future of a technology that will shape everything from medicine to warfare to the global economy. It’s no longer an abstract debate about legal statues. It’s a question about whether the people building this technology can be trusted to slow down before something breaks—and whether the legal system will punish them for trying.
So where does that leave us? Stuck, perhaps, but not stuck forever. The clearest lesson is that Congress needs to act, and it needs to act with more than a bill that sits in committee. Lawmakers need to draw a line between legitimate safety collaboration and illegal collusion, and they need to do it before the next major model release, not after the next major disaster. Antitrust law should not be used as a shield for inaction, and it should not be used as a sword against good-faith efforts to keep people safe. But it’s also true that the industry cannot simply blame the courts for its own failure to work together. OpenAI, Anthropic, Google DeepMind, and others could start having honest conversations today about pacing, safety thresholds, and emergency protocols—they don’t need permission to share research, publish safety findings, or engage in public dialogue. What they need is the will. If they genuinely believe that frontier AI is dangerous, then they should act like it, and they should push Congress to give them the legal clarity they claim to need. If, on the other hand, they are using antitrust concerns as an excuse to avoid tough conversations, then they should be called out for it. The stakes are enormous. We are talking about a technology that could transform the world in ways we don’t fully understand, and the people building it are asking for permission to be careful. That’s not a crazy thing to want. But asking for permission is not the same as taking responsibility. In the end, whether we get safe AI will depend less on legal fine print and more on whether the leaders of these companies are willing to put humanity ahead of their own races, their own egos, and their own bottom lines. That’s a hard ask—but it’s the only way the story ends well.
